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ABSTRACT 

The Experience-Based Career Education (EBCE) model 
being developed and tested in four regions of the United States^ 
under the sponsorship of the National Institute of Education, 
reflects a nationwide interest in discovering new ways to help 
adolescents handle the psychological, social, and economic 
complexities of modern life. This paper reports the attempts to 
integrate and apply diverse research methodologies needed to evaluate 
an individualized experience-based program* Problems encountered in 
undertaking the evaluation of the EBCE Project called Community 
Experiences for Career Education (in Tigard, Ore.) and the results of 
the second-year evaluation there are discussed in this presentation. 
(Author) 
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EVALUATION OF 1HE 
COMMUNITY EXPERIENCES FOR CAREER EDUCATION PROGRAM 



Connnunity Experiences for Career Education — (CE) 

Coxnnunity Experiences for Career Education, (CE)2f is one of four 
Experience-Based Career Education (EBCE) programs being tested under 
the aaspices of the National Institute of Education. Operated in 
Tigc^rd, Oregon by a nonprofit coirmunity corporation, the program has 
been developed by the Northwest Regional Educational Laboratory (NWREL) • 
During the 1973-74 school year (CE) 2 operated with 50 hi^ school 
juniors and seniors and 7 professional staff menbers providing a 
comprehensive high school education through experiences in the conmunity. 

A primary goal of the (CE)2 program has been to integrate a student's 
knowledge of a variety of careers with the acquisition of cognitive, 
interpersonal and affective skills through a series of planned experiences 
with identified learning outcomes. Emphasis is placed on the student 
assundng responsibility for his or her own leamirg. 

Four characteristics, taken together, make (CE)2 different from other 
alternative or career education programs. 

1. The learning program evolves from adult activities in the community. 
It is reasoned that if the learning activities are based directly 

on current adult tasks and roles in the community, the learning goals 
will be recognized by the students as relevant in preparing them for 
the transition to adulthood. 

2. The program is based on experiential learning, meaning tnat students 
are actively involved in the daily work of community life. The 
"hands on" approach to learning has long been recognized as an 
effective learning strategy. E:^erience-based career education is 
attenpting to iinplement this approach in a comprehensive program. 

3. The curriculum of (CE) 2 is fully integrated. Just as the salesman 
or foreman does not think of his interactions with people in terms 
of grammar, vocabulary or psychology, the (CE)2 curriculum also 
applies no c ctificial distinction between the "disciplines." 

4. (CE)2 is a fully individualized program. The learning goals as 
well as the learning strategies are varied to meet the needs, 
interests and abilities of each student. 

The (CE)2 curriculum is divided into three skill areas: Basic Skills, 
Life Skills and Career Development. The Basic Skills area involves the 
traditional "three Rs," reading, mathematics and communication. The 
Life Skills encompass student development in critical thinking, science'; 
perscxial and social developr^ent, functional citizenship and creative 
development and include certification of student competency to perform 
13 "survival skills." Career Developm^t skills include the student's 
acquiring a better knowledge of himself and of career requirements and 
an ability to match the two. 
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A variety of learning strategies are en^loyed to help students attain 
individual objectives under the umbrella of broad curriculum goals in 
the areas of Life Skills, Basic Skills and Career Development. These 
strategies include: individually-negotiated student projects (completed 
on -Tommunity sites as well as at the (CE) 2 learning center); E35>loration 
Packages (guides to help the student investigate new career interests) ; 
learning levels {generally one to four month placements at a particular 
employer site to gain firsthand experience with the job, the employer 
site or a given topic to be es^lored as a project) ; competencies 
(community certified "survival skills") ; weekly student journals (in 
which students communicate in writing with a staff member their observations 
and feelings) ; and tutors and programmed texts as needed to help in iitproving 
Basic Skills. 

Evaluation of the Program 

A coirprehensive formative and summative evaluation of this program has 
been conducted by the EBCE evaluation staff of the Northwest Regional 
Educational Laboratory in Portland, Oregon over the past two years. 
This evaluation, while focusing on student learning, has also examined 
other areas such as program management, and enployer and community 
involveini:nt. This presentation, however, concentrates on the evaluation 
of stvdent outcomes. Evaluation strategies used, methodological problems 
encountered and evaluation findings are presented* 

Because the (CE) 2 project was designed to achieve a wide range of 
cognitive and affective objectives in an individualized manner, the 
evaluators realized that no single research methodology would be adequate. 
Therefore, a combination of various methodologies was employed under the 
assumption that the weaknesses of any one method would be counterbalanced 
by the strengths of another. This combination of research methodologies 
included traditional achievement and attitudinal testing of students in 
a quasi-experimental design, student case studies, unobtrusive nieasures 
and a model borrowed from the legal profession called an adversary 
hearing. 

A quasi-experimental design was used to make comparisons of group data 
from (CE) 2 students with students in selected comparison groups. The 
number of students who volunteered for {CE)2 for the year was not 
sufficiently large to assign students randomly to the experimental or 
control groups and thus precluded the use of a true experimental design. 

Fifty juniors and seniors participating in (CE)2/ all volunteers from 
the local high school, made up the experimental group. 

The comparison groups included: a random sample of 50 juniors and seniors 
from Tigard Higji School (THS) , all the students (25) in the Diversified 
Occupations class of the Cooperative Work Experience (CWE) program at 
Tigard High School and a random sample of 100 students from an occup . ' ional 
skills" center (OSC) in a neighboring district. None of these groups was 
intended to serve as a "control" group in the strict sense of the word, 
but it was felt that each would provide data for meaningful comparisons. 
For exaiiple, the random sample from THS was used to determine the extent 
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to which (CE)2 students w-re similar to, or different from, the "average" 
•IHS student and to collect data with which to compare (CE) 2 student growth 
m Basic Skills. Data from the CWE program was used to determine relative 
growth of (CE)2 and CWE students in career development areas. And data 
from OSC provided conparisons with students in a highly structured 
vocational skills center. 

Data from both standardized and locally-developed instruments were 
collected from the experimental and coirparison groups. They included: 

^' Comprehensive Test of Basic Skins (CTBS) to measure achievement 

m the traditional areas of reading, language, arithmetic and study 
ski lis 

^' the Career Maturity Inventory (CMI) Attitude Scale to measure 
attitudes toward the world of work 

^' Psychosocial Maturity Scale (PSM) to measure miturity in the broad 

areas of individual and social adequacy and interpersonal communicatrons 

'I Student Demographic Questionnaire to record personal background and 
family information, recent activities in high school and in the 
community and reasons for joining (CE)2 

Other survey instruments used to collect data from students in the (CE)2 
program and from their parents and participating employers were: 

^- ^ semantic differential to measure student attitudes toward the 
concepts of Me, Work, Decision Making, School, Learning and Adults 

2- Student Opinion Survey to measure student attitudes toward the 

program 

^' Parent Opinion Survey to measure the attitudes of parents of 

participating students 



4. 



^P-^oyer Questionnaire to collect information on employer 
participation and to measure employer attitudes toward the program 

A case study methodology was employed to integrate the variety of 
information that was being collected and to capture and document the 
program s capability to deal effectively with the individual. The case 
studies of two students were conducted in a "post hoc" manner, utilizing 
data from the extensive recordkeeping system of (CE)2 and supplementing 
these data with interviews with the two students. 

""Qbt^^^ve measures were used to ascertain the extent to 
which different program strategies were being utilized. These measures, 
mostly gleaned from the program recordkeeping system, provided useful 
formative evaluation data on a continuing basis, to the program staff and 
axerted the evaluators to certain issues requiring a more in depth 
investigation. ^ 
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The adversary hearing approach was utilized as an attempt to integrate 
and interpret the evaluation findings in a way maximally tiseful for 
potential adopters of the (CE)2 program. Two advocates collected 
evaluation data and human testimony to support or reject the contention 
that (CE)2 should be adopted by local districts. A prototype of the 
mock trial proceedings, including the testimony and cross-examination of 
four witnesses, was videotaped and used with various audiences. 

Findings 

I 

It was not a^ goal of (CE)2 that xts students make greater growth in Basic 
Skills than a random sample of students at Tigard High School—only that 
they not make significantly less growth. Summary data from the pre- and 
postadministration of the Comprehensive Test of Basic Skills (CTBS) are 
displayed in Table 1.* " 

Table 1 



SUMMARY OF DATA ON PRE- AND POSTADMINISTRATION 
OF THE CC»1PREHENSIVE TEST OF BASIC SKILLS 



CTBS 
Subscore 


Groups Tested 


TiRard 
Juniors N=25 


(CE)2 
Juniors N=10 


(CE)2 
Seniors N=23 


Pre 
X S 


Post 
X S 


Pre 
X S 


Post 
X S 


Pre 
X S 


Post 
X S 


Heading 
Language 
Arithmetic 
Study Skills 


64.5 11.4 

62.6 9.6 
69.4 16.7 
33.0 6.6 


65.9 12.3 
61.4 11.9 
67.0 18.6 
34.6 6.8 


52.4 14.6 
50.0 14.1 
54.4 20.0 
23.0 11.0 


52.3 19.3 
54.2 14.6 
60.0 23.7 
30.5 11.8 


52. 7 13. 1 
51.4 14.0 
47.7 16.0 
26.3 9.2 


56.7 13.2 
53.2 15.0 

56.8 17.0 
30.0 7.5 



* Because of questions concerning the representativeness of the seniors 
in the THS sample, only scores from the THS juniors are presented for 
comparison. 
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A series of four t-tests for matched data was run on the conbined (CE) 
student CTBS data. A sumnary of this analysis is included in Table 2- 

Table 2 



(CE) 2 STUDENT CTBS PRETEST—POSTTEST DIFFERENCES 



Subtest 


Mean 


Gain 


t^ Value 


Pretest 


iVsttest 


Reading 


52.6 


55.3 


2.8 


2.37* 


Language 


51.0 


53.5 


2.5 


1.32 


Arithmetic 


49.7 


57.8 


8.1 


3.98** 


Study Sidlls 


C5. 3 


30.1 


4.8 


2.90** 



• P<.05 
p<.01 



It is noted that statistically significant growth occurred on each CTBS 
subtest except Language. This is especially significant when con^ared to 
(CE)2 student growth in Basic Skills during the first year of program 
activities in 1972-73. During that year no growth in Basic Skills was 
detected using the CTBS. 

To compare (CE) 2 student growth in Basic Skills with that of the comparison 
groups, a univariate analysis of covariance was run on each of the CTBS 
subscores. Pretest scores were used as the covariates. The analysis was 
run comparing THS juniors both with the (CE)2 juniors and with the total 
(CE)2 student population. No significant differences were found in 
either analysis. In other words, the (CE) 2 students gained as much in 
Basic Skills as did the THS students. 

The CMI and PSM were administered to all students in the fall and to 
random halves of each grov^) at midyear and at the end of the year. This 
split schedule of posttesting was implemented to investigate findings 
from the previous year that indicated rapid positive affective change 
during the first semester and relatively little change during the second 
half of the year. The differences between pretest and micfyear for all 
groins on the CMI were almost nonexistent. The overall year change 
(pretest-posttest) is summarized in Table 3. 
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Table 3 



PRETEST AND POSTTEST MEANS AND STANDARD DEVIATIONS 
ON THE CAREER MATUKIT!^ INVENTORY ATTITUDE SCALE 
FOR (CE) 2 AND THE COMPARISON GROUPS 



Scale 


Groups Compared 


{CE)2 
N=15 
Pre Post 


THS 
N=18 
Pre Post 


CSC 
N=30 
Pre Post 


CMI Attitude X 
S 


32.1 36.4 
7.87 4.22 


36.7 36.4 
4.32 7.63 


36.1 38.3 
6.27 4.93 



A t-test for correlated data, run on the {CE)2 students' CMI scores ^ 
yielded a t of 2.71 which is significant beyond the .02 level, indicating 
a positive growth for the year as measured by this instrinnent. Neither 
comparison group ma da a significant change over the year. 

An analysis of covariance {with the pretest score as the covariate) run 
on the data indicated that differential growth among groups was not 
significant. 

A very different pattern of change and one that seemed to confirm the 
earlier stated hypothesis was indicated by the results of the PSM. 
During the first half of the year all groins showed evidence of 
significant positive change in the areas of individual and social adequacy 
and coninuni cation. During the second semester this change was temi>ered 
by a reduction in the rate of growth or by a slxght drop from the midyear 
peak. 

Pretest and posttest scores for the two groups of {CE)2 students tested 
at midyear and at the end of the year are summarized in Table 4. 

Table 4 



SUMMARY DATA ON THE PRETEST / MIDYEAR TEST/ 
POSTTEST OF THE PSYCHOSOCIAL MATURITY SCALE FOR {CE)2 STUDENTS 





Groxjip 


Scale 


Group 1 


Gro\:$> 2 


Pretest Midyear t 


Pretest Posttest t 


Individual Adequacy 


80.6 89.1 2.67 


76.9 87.8 4.38* 


Communication 


78.7 87.3 4.1^ 


75.1 85.1 3.66* 


Social Adeq'^icy 


90. b 108.6 6. of 


97.1 98.9 .44 



*p < .01 
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statistically significant growth was demonstrated on all scales during the 
first half of the year and on the Individual Adequacy and Communication 
Scales over the whole year. 

A multivariate analysis of co variance on the data from all groups did not 
show significant differences between groups in the rate of change but did 
indicate that both the THS and OSC random samples had scores that were 
higher overall than those of the (CE) 2 students. 

A similar pattern of attitudinal change was indicated by a semantic 
differential designed to measure attitudes toward the concepts of Me, 
School, Adults, Learning, Work and Decision Making. This instrument was 
administered to all (CE)2 students in the fall, at midyear and again in 
May. (CE)2 students demonstrated a sicrnificant gain in attitude toward 
each concept between the beginning and end of the first semester. As 
with the PSM, the second semester gains oir the semantic differential 
leveled off. Comparing gains between the beginning and end of the full 
school year, however, revealed significant positive change in attitude 
only towafrd the concepts of School, Me, Learning and Decision Making. 

Questionnaire data from students indicated very positive opinions about 
the (CE)2 program. Only one of the fifty students in the program reported 
that if he had it to do over, he would not participate in this career 
education program. Ninety-six percent of the students felt that, in 
comparison with the regular school program, the (CE)2 program provided 
more opportunity to learn about careers. Seventy-four percent thought 
(CE) 2 provided more opportunity for general learning than did the 
traditional school (21 percent rated it about the same) . 

Parental opinion about the program followed the same trends as student 
opinion. Only one parent indicated some hesitancy about wanting his son 
or daughter to participate in (CE) 2 if he had it to do over (and this 
parent was not sure he would be against the program) . Ninety-six percent 
of the parents thought (CE)2 gave students more opportunity to learn abpUt 
occupations and 92 percent thought it provided more opportunity for general 
learning. And 83 percent of the parents found their son or daughter irore 
motivated to learn in the (CE) 2 program. 

All the participating employers surveyed indicated that they would 
recommend participation in (CE)2 to fellow employers. Personnel at only 
one employer site reported negative reactions by employees to the presence 
of (CE)2 students and about one-half of ^he employers felt the program 
had increased the employees' awareness of youth. In response to a question 
of what students can learn on job sites that they cannot learn in a 
regular classroom, over 50 percent of the employers responding cited the 
students' opportunity to gain first-hand knowledge of job demands in a 
realistic situation. 

Students, staff, parents and employers were each asked to rate the 
importance and, separately, the effectiveness of the program in 
accomplishing 15 student learning outcomes. Each of th^se learning 
outcomes was given an average importance rating of 3 or higher (on a 
5-point scale) by students, staff parents and employ ars— thus indicating 



ERIC 



9 



7 



svpport for the goals of the program. All four groiips considered the 
following learning outcomes to be especially important for (CE) 2 
students: assuming responsibility for themselves, making decisions and 
following through, communicating with others in a mature way, working 
with others, thinking through and solving problems, having a realistic 
attitude toward self, having a positive attitude toward work and learning, 
and inproving interpersonal and social skills. 

The students, staff, parents and employers gave an average rating of 
3 or higher (on a 5-point scale) for effectiveness of 11 of 15 student 
learning outcomes listed. These groves felt the following student 
outcomes were being accoirplished most effectively: performing specific 
occupational skills, assuming responsibility for themselves, communicating 
with others in a mature way, working with others and having a realistic 
attitude toward self. 

Case studies of two (CE)2 students were prepared as a part of the 
evaluation. The purposes of the case studies were to give insights into 
the (CE)2 program that could not be gleaned from quantitative group data 
and to explore hypotheses that might later be tested on a more formal 
basis. 

Three criteria were used to select students for the ^ase study. The 
first criterion specified that one member of each sex be represented. 
The second critericjn required that both students be at least moderately 
successful in the (CE)2 program. This was done to insure that the case 
study describe intended student -program interactions. The final 
criterion required that one student be a junior and one a senior. 

File data were collected on two (CE)2 students: one, whom we shall call 
Kari, was in the program for a year and a half and graduated; and one, 
Mike, spent one year in (CE).2 as a junior. These data (including test 
scores, questionnaires, student projects and resulting products, and 
other performance records) were analyzed and organized by a member of the 
evaluation team. Charts were then prepared to show the chronological 
sequence of activities of each student during the school year. These 
activities, with background and test data, were then organized in 
narrative form. 

Copies of the draft case studies were given to the two students, who 
were asked to read them and verify their accuracy. Interviews with the 
two students were then conducted to fill in any gaps in information and 
to provide more of the human element than could file data. 

Each case study included a description of the background of the student, 
why the student entered the (CE) 2 program, activities and progress while 
in the program and student reactions to (CE)2. 

The first student, Mike, began the (CE)2 school year with low self- 
confidence, deficiencies in reading and communications skills, apd above 
average ability and interest in mathematics. His long range goal was to 
become a computer operator. 
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Mike's early (CE) 2 experiences were marked by tardiness and lack of 
commitment. He did not apply himself well at an elementary school where 
he worked for three and one-half months. During the second semester, 
however, he had the opportunity to work at a local bank where he dealt 
firsthand with computers and computer programming. His attitude and 
performance changed markedly. His motivation and punctuality improved 
drastically. He read about conputers and worked closely with his 
employer instructor. Both his self-confidence and his oral communication 
skills improved. 

The Life Skills projects that Mike coitpleted during the year were designed 
to alleviate his lack of communications skills and self-confidence and to 
build on his interest in math and technology. His Canadian heritage 
became the impetus for a project on immigration laws and hi3 German 
ancestry influenced him to take a course in German at Tigari High School. 

Mike felt that his big accomplishments for the year included a strengthened 
interest in computer technology and determination to find a career in that 
area. His pe"-sonal appearance and self-confidence in^roved greatly and 
he reports with pride that he can now speak up in groups. And, while 
prior to (CE)2 Mike never did unrequired reading but found TV his favorite 
pastime, he reports that he read eight books this past summer and has 
begun a hobby of collecting computer books and materials. 

The other student, Kari, an above average student at Tigard High School, 
entered (CE)2 primarily because she saw the traditional high school 
curriculum as irrelevant. Despite opposition by her parents and friends, 
she was determined to try the (CE) 2 alternative. 

Kari's initial experiences in the program fulfilled her worst expectations. 
She was bored by the initial inactivity and frustrated by the often 
detailed requirements of the program. She remained critical of many of 
the requirements of the program although she finished all requirements 
for the first year and returned for the second despite parental pressure 
to return to a more traditional high school setting. Her parents felt 
the program needed more organization. 

Her second year at (CE)2 was marked by fluctuations both in her mood and 
performance. She began the year with enthusiasm and completed seven 
projects during the fall semester. At midyear, however, she again became 
disenchanted with what she was doing and har productivity dropped off. 
A noncommittal attitude and lack of follow through on her commitments 
were observed by the staff- A number of staff counseling sessions were 
held with her to help her better cope with various changes in her life. 
By February she had regained her excitement and became heavily involved 
in her learning level exq>eriences . 

Kari's big accoit«)lishment at (CE)2. in her opinion, was her "growing up." 
Both she and the staff recognized in retrospect that her ups and downs 
were symptoms of. the normal transition from teenager to adulthood. They 
feel that she xnade the transition well. Kari's second major accomplishment 
was a decision about a career. At the beginning of her senior year she 
was undecided about further education beyond high school but her 
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experiences solidified in her mind her interest in secretarial work and 
in work as a telephone operator. After completing her (CE)2 program 
requirements in early May she began a permanent position with the 
telephone con^any as a telephone operator, a position in which she gained 
experience on one of her learning level placements. 

Conclusion 

(CE)2 has now completed two full years of operation as an alternative 
career education program for high school juniors and seniors. This 
approach provides an integrated and comprehensive learning program 
through individualized activities both in a learning center and in the 
community. The main tenet of the program is that learning through 
direct experience is more effective than learning through v*.cario\xs 
experience. 

The prograiH has demonstrated important accomplishments. Program 
participants have reacted favorably toward it. The students see the 
program as relevant and exciting and have demonstrated a renewed interest 
in their own education. Parents confirm this new level of motivation 
exhibited by their sons and daughters and are impressed by the learning 
opportunities made availcible to them. Participating employers have 
fo'jnd the program beneficial both to the students and to the participating 
con5>auiies and, on the jfhole, give their enthusiastic support. 

Measures of student outcomes from the program are also encouraging. 
Students made significcmt gains in Basic Skills and when (CE)2 scorers 
were conpared with regular high school scores, there was no evidence 
that this alternative experience-based program penalized the student 
in reading, arithmetic or communication. Students also demonstrated 
significant growtii in their attitudes toward themselves and in measures 
of individual adequacy. They also gained in their relationships and 
interactions with others. They became more positive toward school, 
learning and decision making, and in the area of career maturity 
demonstrated a significant positive change in their attitude toward work. 

The program is currently in its third year of operation at Tigard and 
preparations are now being made to add, through local district funding, 
other replication sites in three states of the Pacific Northwest. 
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